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Typically, models with a heterogeneous property are considerably 
harder to analyze than the corresponding homogeneous models, in 
which the heterogeneous property is replaced with its average value. 
In this study we show that any outcome of a heterogeneous model 
that satisfies the two properties of differentiability and interchangi- 
bility, is 0(e 2 ) equivalent to the outcome of the corresponding ho- 
mogeneous model, where e is the level of heterogeneity. We then 
use this averaging principle to obtain new results in queueing theory, 
game theory (auctions), and social networks (marketing). 

mathematical modeling | heterogeneity | averaging | homogenization 



model that satisfies the two properties of differentiability and 
interchangeability, is 0(e 2 ) asymptotically equivalent to the 
outcome of the corresponding homogeneous model, i.e., 

F(jj,i, . . . ,/ifc) = Jhomog. (/i) + 0{e 2 ). 

Thus, if the function F is also interchangeable, the error 
of the approximation JTJ for a 10% heterogeneity level is 
only 0(1%). 



Mathematical modeling is a powerful tool in scientific re- 
search. Typically, the mathematical model is merely an ap- 
proximation of the actual problem. Therefore, when choosing 
the model to work with, one has to strike a balance between 
complex models that are more realistic, and simpler models 
that are more amenable to analysis and simulations. This 
dilemma arises, for example, when the model contains a het- 
erogeneous quantity. In such cases, a huge simplification is 
usually achieved by replacing the heterogeneous quantity with 
its average value. The natural question that arises is whether 
this approximation is "legitimate", i.e., whether the error that 
is introduced by this approximation is sufficiently small. 

Let us illustrate this with the following example, which 
will be discussed in details later on. Consider a queue 
with k heterogeneous servers, whose expected service times 
are {in, ■ ■ ■ , Hk}- We would like to calculate analytically 
the expected number of customers in the system, which 
we denote by F(pi, . . . , Atfc)EI While an explicit expression 
for F(fii, . . . is not available, there is a well-known ex- 
plicit expression in the case of k homogeneous servers, which 
we denote by -fh omog .(/i) := F([i, . . . , p). A natural approxi- 

Xk 

mation for the expected number of customers in the system 
is 

F((J,1, . . . ,Hk) « Fhomog.(/x), [1] 

where fi is the average of {/ii , . . . , /Ufc}. 

More generally, let F(jjli, . . . ,/ife) denote an "outcome" of 
a heterogeneous model, let 



maxi<;<fc \fj,i 



[2] 



denote the level of heterogeneity of {pi, . . . , fj,k}, and let 
fiiomog. (m) denote the outcome of the corresponding homo- 
geneous model. If the function F(jmi, ■ ■ ■ , fJ.k) is differentiable, 
then it immediately follows that 



, Hk) = fiiomog. (m) + °(e). 



Therefore, roughly speaking, for a 10% heterogeneity level, 
the error of approximating F(fii, . . . , fik) with Fhomo g .(/i) 
is 0(10%). In many studies in different fields, however, re- 
searchers have noted that the error of this approximation is 
considerably smaller than O(e). Moreover, this observation 
seems to hold even when the level of heterogeneity is not small. 

In this study we show that these observations follow from 
a general principle, which we call the Averaging Principle. 
Specifically, we show that any outcome of a heterogeneous 



The Averaging Principle 

Let F(/j,i, . . . ,/ifc) be an outcome of a model with a hetero- 
geneous property, captured by the k parameters {fii, . . . , fJ>k}, 
that satisfies the following two properties: 

1. Differentiability: F is twice-differentiable at and near 
the diagonal fii = ■ ■ ■ = [Xk- 

2. Interchangeability: For every (fii, . . . , /xj.) £ R fe and ev- 
er y i 7^ jj F(. . . , fii, . . . , fij, . . .) = F(. . . , fij , . . . , fii, . . .). 
Thus, the outcome F is independent of the identi- 
ties/indices of the heterogeneous parameters^ 

Then, we have the following result^ 

Theorem 1. (The Averaging Principle) Let F satisfy the differ- 
entiability and interchangeability properties. Let fi — 
(fii, . . . ,fik) be "sufficiently close to the diagonal", i.e., 



where /x 4 = (ji/ 



Im-maII < Caa. 

,P.a), P-a = j Si=i W * s ^ e arithmetic 



average, || ■ || is a vector norm on R , and Cp A is a positive 
constant that only depends on JIa (and of course on F). Then, 

F(m, . . . , Hk) = Fhomog.(MA) +0(\\n - AaII 2 )- [3] 
where F holnog .(fJ>) := F(/j,, . . . , /i) . 

Theorem [1] remains valid if {/ii, . . . , pifc} are functions and 
not scalars, see the Game Theory (auctions) example below. 



Reserved for Publication Footnotes 



1 In order to focus on the heterogeneous property, we suppress the dependence of F on other 
parameters. 

2 For example, in the queueing-system example, switching the identities/locations of two servers 
does not affect the expected number of customers in the system. 

^ This and all other proofs are given in the Appendix. 
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Using the geometric and harmonic averages. In Theorem [T] 
we averaged the {fn}s using the arithmetic mean. It is well- 
known in homogenization theory that in some cases the cor- 
rect homogenization is provided by the geometric or the har- 
monic mean. To address the question of the "correct" aver- 
aging, we recall the following result: 

Lemma 1. Let /j, > 0, and let {hi,...,hk} 6 K. Then, 
as e — > 0, the arithmetic, geometric and harmonic means 
of {/! + ehi, . . . , fi + ehk} are 0(e 2 ) asymptotically equivalent. 
Proof. We can prove this result using the averaging prin- 
ciple. Let fiA denote the arithmetic mean of {/i + eh\ ,fi-\- 
ehk}. The geometric mean /xg(/x + ehi,...,fJ, + ehk) = 
Ik \ x l k 

( Yli=i (m + ) satisfies the interchangeability and dif- 
ferentiability properties. Therefore, application of Theorem [T] 
gives 



MG , ■ ■ ■ , V+eh k ) = fi G (ha 



,p A )+0(e 2 ) = p A +Q(e 2 



The proof for the harmonic mean fin = fc/(^- + -- - + ^=-)is 
similar. □ 



From Lemma [T] and the differentiability of Fhomog. (m) it 
follows that 



-Fkomog. (a*a) 



-Fhomog.(AtG) + 0{e 2 ) 
F honlog .(p,n) + 0(e 2 ). 



Corollary 2. Let fi > 0. Then, the averaging principle (Theo- 
rem Qp remains valid if we replace the arithmetic mean with 
the geometric or harmonic means. 

A natural question is which of the three averages is "op- 
timal", in the sense that it minimizes the constant in the 
0(\\fi — /i|| 2 ) error term. The answer to this question is model 
specific. It can be pursued by calculating explicitly the 0(e 2 ) 
term, as we will do later on. 

Weak Interchangeability. To extend the scope of the averaging 
principle, we define a weaker interchangeability propertylj 

2A. Weak interchangeability: For every fi, fi and every 
1 < jo < k, if j-ij = fi and fij — fi for all j /= jo, then 
F(fi\, . . . , is independent of the value of jo- 

Thus, F(/j,i,--- is weakly interchangeable if, whenever 

all but one of the parameters are identical, the outcome F is 
independent of the identity (coordinate) of the heterogeneous 
parameter. 

Every interchangeable function F is also weakly inter- 
changeable, but not vice versa. Nevertheless, the proof of 
Theorem [T] implies that: 

Corollary 3. The averaging principle (Theorem Qp remains 
valid if we replace the assumption of interchangeability with 
the assumption of weak interchangeability. 

Queuing theory application: An M/M/k queue with 
heterogeneous service rates 

Consider a system with k servers. Server i has a random 
service time that is distributed according to an exponential 
distribution with rate fit. Customers arrive randomly accord- 
ing to a Poisson distribution with arrival rate A. An arriving 
customer is randomly allocated to one of the non-busy servers, 
if such a server exists. Otherwise, the customer joins a waiting 
queue, which is unbounded in length. Once a customer is allo- 
cated to a server, he gets the service he needs and then leaves 
the system. This setup is known in the Queuing literature 



as M/M/k modelQ Examples for such multi-server queuing 
systems are call centers, queues in banks, parallel computing, 
and communications in ISDN protocols. 

Let F{p,\, . . . , fj,k) denote the expected number of cus- 
tomers in the system (i.e., waiting in the queue or receiv- 
ing service) in steady state. In the case of two heterogeneous 
servers, F(fii, [12) can be explicitly calculated (see Appendix): 

Lemma 2. Consider an M/M/2 queue with heterogeneous 
servers. The expected number of customers in the system is 
given by 



F(ni,fi 2 ) = 



P ~ 



v ' ' p (H1+M2) 2 i-p 

[4] 

Finding an explicit solution for F(fii, ... ,{ik) when k > 3 
is computationally challenging, because it involves solving a 
system of 2 k — 1 linear equations. In the homogeneous case 
fj,i = • • • = fj,k = however, it is well-known that 



F(p,...,n) 



fe-1 (Vri" 



Ek — 1 
n = 



+ 



(A/M) fc 



kjj. 



A 
/'' 



The function F(fii, . . . , fi k ) can be written as a sum of so- 
lutions of a system of linear equations with coefficients that 
depend smoothly on fii , . . . , fik (see the appendix). Therefore, 
F is differentiable. Since customers are randomly allocated to 
the free servers, renaming the servers does not affect the ex- 
pected number of customers in the system. Hence, F is also 
interchangeable. Therefore, we can use the averaging principle 
to obtain an explicit 0(e 2 ) approximation for F(/ii, . . . , fik): 
Theorem 4. Consider an M/M/k queue with heterogeneous 
servers whose service rates are {fii, ■ ■ ■ , Hk}- The expected 
number of customers in the system is given by 



F(fj, 1 , ...,ii k ) = Jhomog. (P) + 0(e 2 ), 



where F\ 



homog 



(/') 



F(fj,, . . . , /Li) is given by f5||. /1 



£ 53i=i Mi> an d e ls 9™ e by [EJ. 

For example, by Theorem [4] the expected number of cus- 
tomers with 2 heterogeneous servers is 

F( Ml , M2 ) = F(A,M) + 0(e 2 ) " ' A/ ' 
where 



A* = 



Mi + [1.2 



4 p? - A 2 

fl 2 — Ml 



+ 



Indeed, substituting fj,i t 2 = p, ± e in the exact expression ]4j 
and expanding in e gives [6j] . 

In the case of k = 8 heterogeneous servers, even writing 
the system of 2 s — 1 = 255 equations for the 255 unknowns is 
a formidable task, not to mention solving it explicitly. By the 
averaging principle, however, 

F(pi,. ..,p 8 ) = i*k,mog. (a*) + O (e 2 ) , 

where Fkomog. (m) := F(p,, . . . ,p.) is given by [5[ with k = 8. 

X8 

We ran stochastic simulations of an M/M/8 queuing system 
with 8 heterogeneous servers using the ARENA simulation 



See the social-networks application below for an example of a weakly-interchangeable outcome 
which is not interchangeable. 



3 For an introduction to queueing theory, see e.g., [6], 
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Fig. 1. The relative error of the averaging-principle approximation for the steady- 
state number of customers in a system with 8 heterogeneous servers, as a function 
of the heterogeneity parameter e. The solid line is error = 0.594e 2 . The crosses 
denote the relative error of the improved approximation II15II . The dotted line is 
error = 0.074e 3 . 



software, and used it to calculate the expected number of cus- 
tomers in the system. The simulation parameters were 



28 
hour ' 



I' 



hour 



fii = fi + shi, 



(hi,...,h a ) = (1, 1.5, 2, 3, 3.5, -2.5, -4, -4.5) 



hour 



and e varies between and 1 in increments of 0.05. Because 
J2i=i hi = 0, the average service rate is \x = fx = 5. Therefore, 
by Theorem Q] 

F(fx + ehi , . . . , n + ehs) = F houlog . (5) + O (e 2 ) . 

In addition, by Equation [5fl, -Fhomog. (5) = 6.2314. 

To illustrate the accuracy of this approximation, we plot 
in Fig. Q]the relative error of the averaging-principle approx- 



imation 



F(n+eh 1 ,...,n+eh s )-F ha 



.(5) 



— — , — , . As expected, this error 

scales as e 2 . Note that even when the heterogeneity is not 
small, the averaging-principle approximation is quite accu- 
rate. This is because the coefficient (0.594) of the 0(e 2 ) term 
is small|3 For example, when e = 0.5 the relative error is 
« 2%, and for e = 1 it is below 10%. 

Remark. We can also use the averaging principle to ob- 
tain 0(e 2 ) approximations of other quantities of interest that 
satisfy the interchangeability property, such as the average 
waiting time in the queue, or the probability that there are 
exactly m customers in the queue. 



Game theory application: Asymmetric Auctions 

Consider a sealed-bid first-price auction with k bidders, in 
which the bidder who places the highest bid wins the ob- 
ject and pays his bid, and all other bidders pay nothingQ A 
common assumption in auction theory is that of independent 
private-value auctions, which says that each bidder knows his 
own valuation for the object, does not know the valuation of 
the other bidders, but does know the cumulative distribution 
functions (CDF) of the valuations of the other bidders. Bid- 
ders are also characterized by their attitude towards risk: The 
literature usually assumes that bidders are risk neutral, since 
this simplifies the analysis. More often than not, however, 
bidders are risk averse. 

A strategy of bidder i is a function bi(-) that assigns a 
bid bi(vi) to each possible valuation Vi of that bidder. The 
bid that a bidder places depends on his valuation Vi, and on his 
beliefs about the distributions of the valuations of the other 
bidders and about their bidding behavior. An equilibrium in 
this setup is a vector of k strategies {bi(-)}i = i, such that no 



single bidder can profit by deviating from his bidding strat- 
egy, whatever his valuation might be, so long that all other 
bidders follow their equilibrium bidding strategies. 

Most of the auction literature focuses on the symmetric 
(homogeneous) case, in which the beliefs of any bidder about 
any other bidder (e.g., about his distribution of valuations, his 
attitude towards risk, etc.) are the same. In this case, one can 
look for a symmetric equilibrium, in which all bidders adopt 
the same strategy. In practice, however, bidders are usually 
asymmetric (heterogeneous), both in their attitude towards 
risk and in the distribution of their valuations. Each bidder 
then faces a different competition. As a result, the equilibrium 
strategies of the bidders are not the same. 

The addition of asymmetry usually leads to a huge com- 
plication in the analysis. For example, in the case of a 
first-price auction for a single object with risk-neutral bid- 
ders that have private values that are independently dis- 
tributed in the unit interval [0, 1] according to a common 
function F(v), the symmetric Nash equilibrium inverse bid- 
ding strategy v(b) = b -1 (t>) satisfies the ordinary differential 
equation (ODEJ 



v'(b) = 



1 F(v(b)) 



k - 1 F'(v(b)) v(b) - b' 
This equation can be solved explicitly, yielding 

J v F"-\s)ds 



v(0) = 0. 



b(v) 



j?fc-i(„) 



[7] 



Therefore, this case is "completely understood". From the 
seller's point of view, a key property of an auction is his ex- 
pected revenue. In the symmetric case, the expression [7[ can 
be used to calculate the seller's expected revenue -Rhomog. [F], 
yielding 



-Rho 



[F] = l + (k-l) f F k (v)dv-k [ F fe_1 (u) 
Jo Jo 



dv. [8] 



In the asymmetric case, where the value of bidder i is inde- 
pendently distributed in [0, 1] according to Fi(v), the inverse 
equilibrium strategies {vi(-)} , [= 1 are the solutions of the sys- 
tem of ODE's 



v'i(b) 



Fi(vi{b)) 



\k-lf^(v,(b)-b) ) 



(Mb) - b) 



[9al 



for i = 1, • • ■ , k, subject to the initial conditions 

Vi (b = 0)=0, i = l, [9b] 

and the "end condition" at some unknown b 



Vi(b) = 1, 



i = l, 



[9cl 



Thus, the addition of asymmetry leads to a huge complica- 
tion of the mathematical model: instead of a single ODE that 
can be explicitly integrated, the mathematical model consists 
of a system of coupled nonlinear ODE's with a non-standard 
boundary condition. As a result, the system [9] cannot be 
explicitly solved, and it is poorly understood, compared with 
the symmetric case. 

In [2] , Fibich and Gavious considered the system ]5J in the 
weakly-asymmetric case Fi = F + eHi, i — 1, . . . , k. After sev- 
eral pages of perturbation-analysis calculations, they obtained 



6 This coefficient will be computed analytically later on from eq. II 1411 . 
7 See [7] for an introduction to auction theory. 

^Since we consider the case where all bidders use the same strategy, we omit the subscript i from b 
and v. 
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0{e 2 ) asymptotic approximations of the inverse equilibrium 
strategies {vi(b; e)}i=i . Substituting these approximations in 
the expression for the seller's expected revenue, showed that 
it is given by 

R[Fi=F + eHi,...,F h = F + eH k ]=Rhomo S .[F} [10] 
-e(k - 1) [\l - F(v))F k - 2 (v) V Hi(v) dv + 0(e 2 ). 

Subsequently, Lebrun [8] proved that the function on the 
left-hand-side of JTD] is differentiable in e, and used that to 
show that Eq. | ll()l | holds. This is, in fact, a special case 
of the averaging principle. Indeed, interchangeability holds 
since changing the indices of the bidders does not affect the 
revenue, and, as mentioned above, differentiability in e was 
proved in [8]. Therefore, by the averaging principle for func- 
tions (see the appendix), 

R[F 1 =F + eHi,...,F k = F + eH k ] = R homog . [F] + 0(e 2 ) 

[11] 

where F = F + %J2i=i H i- Substituting F in [U and ex- 
panding in powers of e gives 



Rh. 



omog. [F]=R nomog. [F] 
e(k - 1) f\l - F(v))F k ~ 2 (v) Ht(v) dv + 0(e 2 







i=l 



Hence, relation | |10| | follows. 

Numerical calculations (0 Table 1] and |31 Tables 1-2]) 
show that the error of the averaging-principle approxima- 
tion [1111] is small (typically below 1%), even when the asym- 
metry level is mild (e.g., e = 0.4). This provides another il- 
lustration that the averaging-principle approximation can be 
useful even when e is not very small. 

The averaging principle does not only lead to a simpler 
derivation of relation | ll()l | . but also enables us to derive a 
more general novel result: 

Theorem 5. Consider an anonymous a?trfio?|3 in which all k 
bidders have the same attitude towards risk, and all bid- 
ders follow the same "rules" when they determine their bid- 
ding strategies]^ Let F\, • ■ • , F k be the cumulative distri- 
bution functions of the valuations of the bidders, and let 
R[Fi,--- ,Fk] be the expected revenue of the seller. If R is 
twice differentiable at and near the diagonal, then 

R[F U ■■■ ,F k ] = R homos . [F] + 0{e 2 ), 

where -Rhomog. [F] — R[F, • • ■ ,F], F is the average 
of Fi, ■ ■ ■ ,Fk, and e is the level of heterogeneity. 

Indeed, the assumptions of the theorem imply that F is 
interchangeable. Therefore, if F is also differentiable, the the- 
orem follows from the averaging principle. 



structure, since adopters can only influence people that they 
"know". The social network is usually modeled by an undi- 
rected graph, where each vertex is an individual, and two 
vertices are connected by an edge if they can influence each 
other. 

The first quantitative analysis of diffusion of new products 
was the Bass model [I], which inspired a huge body of theo- 
retical and empirical research. In this model and in many of 
the subsequent product-diffusion models: 

1. A new product is introduced at time t = 0. 

2. Once a consumer adopts the product, he remains an 
adopter at all later times. 

3. If consumer j has not adopted before time t, the probabil- 
ity that he adopts the product in the time interval [t,t + s), 
given that the product was already adopted by n, (i) peo- 
ple that are connected to j, and that no other consumer 
adopts the product in the time interval [t, t + s), is 

no other consumer 
adopts in [t,t + s) 



Prob [ j adopts in [t,t + s)jnj(t) 
rij (t) 



= [Pj + 



■q 3 )s + Q(s z 



[12] 



as s — > 0, where rrij is the total number of individuals 
connected to consumer j and the parameters pj and qj de- 
scribe the likelihood of individual j to adopt the product 
due to external and internal influences, respectively. 

We say that a social network is translation invariant, if any 
individual sees exactly the same network structure. Therefore, 
in particular, mj is independent of j. Examples of translation- 
invariant social networks are (see Fig. [2]): 

A) A complete graph, in which any two individuals are con- 
nected. 

B) A one-dimensional circle, in which each individual is con- 
nected to his two nearest neighbors. 

C) A one-dimensional circle, in which each individual is con- 
nected to his four nearest neighbors. 

D) A 2-dimensional torus, in which each individual is con- 
nected to his four nearest neighbors. 

We say that all individuals are homogeneous when all 
individuals share the same parameters, i.e., pj — p and 
Qj — q for every individual j. Let N(t) denote the num- 
ber of adopters at time t. The expected aggregate adoption 
curve -Ehomog. [N(t; p, q)] in several translation-invariant social 
networks with homogeneous individuals were analytically cal- 
culated in [101 14]. In these studies, the assumption that all 
individuals are homogeneous was essential for the analysis. 

One of the fundamentals of marketing theory is that 
consumers are anything but homogeneous. An ex- 
plicit calculation of the expected aggregate adoption curve 



Social-networks application: Diffusion of new products 

Diffusion of new products is a fundamental problem in Mar- 
keting, which has been studied in diverse areas such as retail 
service, industrial technology, agriculture, and educational, 
pharmaceutical and consumer-durables markets [9]. Typi- 
cally, the diffusion process begins when the product is first 
introduced into the market, and progresses through a series 
of adoption events. An individual can adopt the product 
due to external influences such as mass-media or commer- 
cials, and/or due to internal influences by other individuals 
who have already adopted the product (word of mouth). The 
internal influences depend on the underlying social-network 







A B C 

Fig. 2. Examples of translation-invariant networks. 



D 



i.e., an auction in which the winner and the amount that each bidder pays depend solely on their 
bids, and not on the identity of the bidders. 

10 For example, bidders may use bounded rationality |11| when determining their bidding 
strategies. Thus, bidders may restrict themselves to a class of simple strategies, such as low-order 
polynomial functions of the valuation v. They may even not be aware of the concept of equilibrium. 
Nevertheless, as long as all bidders have the "same" bounded rationality, the interchangeability 
requirement holds. 
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E[N(t;{pj},{qj})] in the heterogeneous case, however, is 
much harder than in the homogeneous case. As a result, the 
effect of heterogeneity is not well understood. 

The averaging principle allows us to approximate the 
heterogeneous model with the corresponding homogeneous 
model. Consider a translation-invariant network. Then, 
for t > the function F({pj}, {q 3 }) := E[N {t; {pj} , {q 3 })] 
is differentiate and weakly-interchangeable (see Appendix). 
Therefore, by the averaging principle, 

Theorem 6. The expected aggregate adoption curve in a trans- 
lation invariant social network with heterogeneous individuals, 
can be approximated with 

£[iV(t;fe}, {&•})] = E homogcncous [N(t;p,q)} +0(e 2 ), 

where p and q are the averages of {pj} and {qj}, respectively, 
and e is the level of heterogeneity of {pj} and {qj}. 

Theorem [6] is consistent with previous numerical findings: 

• In [5], simulations of an agent-based model with a complete 
graph showed that heterogeneity in p and q had a minor 
effect on the expected aggregate adoption curve. 

• Simulations of agent-based models with ID and 2D 
translation-invariant networks 'A, Figure 18] showed that 
when the values of {pj } and {qj } are uniformly distributed 
within ±20% of the corresponding values p and q of the 
homogeneous individuals, the heterogeneous and homoge- 
neous adoption curves are nearly indistinguishable. Even 
when the heterogeneity level was increased to ±50%, the 
two adoption curves were still very close. 



Calculating the O(c^) term 



The averaging principle is based on a two-term Taylor expan- 
sion of F. Therefore, the error of this approximation is given, 
to leading order, by the quadratic term in this expansion. 
When F satisfies the differentiability and interchangeability 
propertied]] and /i is the arithmetic mean, this error is given 
by (see the appendix): 



Therefore, this calculation only requires the explicit calcula- 
tions of F in the homogeneous case /ii = • • • = /ifc, and in the 
case that the heterogeneity is limited to a single coordinate 
(i.e., when \i2 = ■•■ = Mfe). In many cases, this is a con- 
siderably easier task than the explicit calculation of F in the 
fully-heterogeneous case. 

To illustrate this, consider again the M/M/k example of 
Fig. [T] with k — 8. While the fully-heterogeneous case re- 
quires solving 2 k — 1 = 255 equations, the single-coordinate 
heterogeneous case requires solving only 2 • k = 16 equations. 
Solving these 16 equations symbolically and using Lemma [3] 
yields, see the appendix, 



a(k = 8) = 



[141 



2Xfl (EUMfr) 2 

where the values of {d,fo} are listed in the following table: 



i 


Ci 


h 





1 


1 


1 


45 


14 


2 


999 


126 


3 


14280 


840 


4 


144720 


4200 


5 


1088640 


15120 


6 


6249600 


35280 


7 


27941760 


40320 


8 


97977600 




9 


263390400 




10 


514382400 




11 


653184000 




12 


406425600 





In particular, substituting /i = 5 and A = 28 yields a 
0.00837. This leads to the improved approximation 



F(fl!,. . . ,/ifc) 

where 



1 



F(flA, 



d 2 F 



2 \ djj,id)ii 



d 2 F 



[13al 



[13bl 



Therefore, 



1. The magnitude of this error is ~ \a\ 1 1 /j. — /i 4 || . 

2. The sign of this error is the same as the sign of a. 

A Taylor expansion in h gives 
F(fiA + 2h, [ja, a*a, • ■ • , Ma) - F(/ia +h,p,A + h, p-A ■ 



,Ma) 



2ah 



This shows that in order to determine the sign of a, one can 
compare the effect of adding h units to two parameters with 
the corresponding effect of adding 2h units to a single param- 
eter. 

The value of a can be calculated as follows: 
Lemma 3. Assume that F(fii, . . . .p,k) satisfies the differentia- 
bility and interchangeability properties. Then, 



k 



2(k- 



2 F 



A,- 



1 E 



homog 



(ma) 



F(m, . . . ,ns) ~ fhomog.M + ^y^Mi - m) 2 

1=1 

« F homog .(5)±0.594e 2 . [15] 

The error of this improved approximation scales as 0.074e 3 , see 
Fig. [T] which is the next term in the Taylor expansion. In par- 
ticular, the relative error of | |15| | is below 1.5% for < e < 1. 



Final remarks 

The averaging principle is based on a simple observation: the 
leading-order effects of heterogeneity cancel out when the out- 
come is interchangeable. Nevertheless, it can lead to a signif- 
icant simplification of mathematical models in all branches of 
science. The averaging principle is unrelated to averaging that 
originates from laws of large numbers in large populations, and 
it holds, e.g., when there are few servers in a queuing system, 
or a few bidders in an auction. 

The interchangeability and the weak interchangeability 
properties are usually easy to check. The differentiability of F 
is easy to check in some cases, but can be quite a challenge in 
others. We note, however, that more often than not, functions 
that arise in mathematical models are differentiable, unless 



Here we cannot assume that F is only weakly interchangeable, since we require that ^ 



for all i, j. 
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there is a "very good reason" why they are not. While this is 
a very informal statement, we make it in order to point out 
that the "generic" case is that the outcome F is differentiable, 
rather then the other way around. 

An important issue is the "level of heterogeneity" that is 
covered by the averaging principle. Strictly speaking, the level 
of heterogeneity should be "sufficiently small". In practice, 
however, in many cases the averaging principle provides good 
approximations even when e = 0.5. In other words, the coeffi- 



cient of the 0(e 2 ) term is O(f). While this is also an informal 
statement, we make it in order to point out that one should 
not be "surprised" that the averaging principle holds even 
when e is not very small. 
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Fig. 3. Transition diagram of a queue with two heterogeneous servers. State "0" 
corresponds to the situation in which no server is busy. States (1,0) corresponds 
to the situation in which server 1 is busy and server 2 is not busy. States (0, 1) 
corresponds to the situation in which server 1 is not busy and server 2 is busy. State 
"fc" for k > 2 corresponds to the situation in which both servers are busy and k — 2 
customers wait in the queue. 




Fig. 4. Same as Fig. [3] with three heterogeneous servers. For example, state 
(0, 1, 1) corresponds to the situation in which server 1 is not busy and servers 2 
and 3 are busy. 



the following equalities hold: 



A (1,0) . M 

\po = mpi + pip\ 

-Pa + pipi = ( A + pi )Pi 1,0) , 
-Po + pipi = (A + ^ 2 )Pi 0,1) , 



.(0,1) 



[16a] 
[16b] 

[16cl 



Ap< M) + Ap< 0,1) + (pi+p 2 )p S = (A + Ml + Pl)Pl, [16d] 

Ap„ + (/J,! + pi)p n + 1 

= (A + pi + M2)Pn+i, n = 2, 3, ... 
We can view |16a(] — [ fl6c(] as a linear system for the three 
unknowns {po,Pi ,Pi }■ Solving this system for po yields 



[16el 



Po = 



Ipipi 

A 2 



-pi. 



In addition, the solution of | |16d| |-| fl6e| | is p n = 

\ n —2 

— r — ) P2 = p n ~ Pi for n > 1. Substituting the above 

Ml "rA i 2 / 



~>n = P0 
n—0 n—1 



gives pi= (2iii^ + I-L 



P2 



P 1-P 



A 2 

Therefore, 



2jLtl)U2 1 1 



P 1 - P 



Pi, 



Appendix 



Proof of Theorem [T] 

Because of the differentiability of F, there exists a positive 
constant Cp, A , such that for all \\p, — £L A \\ < Cp, we can ex- 
pand F((jl) as 



3=1 

Because F is interchangeable, 
dF 



dF 
dfij 



•0(||m-AaI 



A 



cfyti 



A., 



Therefore, 



X](aj -ma) + o(||m-MviI 

A A 3=1 



Since is the arithmetic average, X^=i(P:i ~ P A ) = 0. 
Hence, the result follows. 



F(pi,p 2 ) = ^2 np n = ^ np n 2 pi = —^2 n P" 1 

n=0 n = P n=0 



P2 ^ „ = P2 1 ) - P2 1 
and the result follows. 



M/M/3 queue 

Consider the case of three heterogeneous servers with aver- 
age service times pi, pi and p3. Denote by po, Pi*' ' ', p^ ' 1,0 ', 
pj 1,1 ' ', p^' 0,1 \ Pa ' 1 ' , P3, P4, the steady-state 
probabilities. Thus, for example, p^ 1 ' 0,1 ' is the steady-state 
probability that servers 1 and 3 are busy, server 2 is free, and 
there are no waiting customers in the queue (we denote by 
p„,n > 2 the probability having n customers in the system) . 
The transition diagram for k — 3 servers is given in Fig. [4] 
The steady-state equations are 

> (1,0,0) . (0,1,0) . (0,0,1) 

Apo = pip\ + p 2 p\ + p 3 p\ , 



A (1,1,0) . (i,o,i) 

-P0 + PlP\ ' + PZP 2 

A (i,i,oi , (0,1,1) 

-P0 + piP 2 + M3P 2 

A 



(pi + \)p[ lfifi \ 
(pi + X)p^°\ 



(1,0.1) . (0,1,1) / . \\ (0,0,1) 

^po + pip 2 +M2P2 =(M3 + A)pi 
A (1,0,0) , A (0,1,0) . / \ , \ (1,1,0) 

TTPi + nPl + P3P3 = (A + fll + p 2 )p 2 



.(1,0,0) 



- (0,0,1) . /, , \ 

0P1 + 2 p i +M2P3 = (A + Mi +M3)P 2 

A (0,1,0) A (0,0,1) > (0,1,1) 

ttPi +2^1 +M1P3 = (A + M2 + M3)P 2 

= (A + m + ^ 2 + M3)p3, 

Ap„ + (m + pi + p- A )p n+ i 

— (A + pi + pi + p s )p n+ i, n > 3, 



(1,0,1) 



Proof of Lemma [5] 

We calculate F(pi,pi) explicitly using the steady-state 
transition diagram that is shown in Fig. [3] We denote by pi 
the steady-state probability for the system to be with i cus- 
tomers, and by p^ 1 ' ' and p^ 0,1 ' the steady-state probability 
for the system to be with 1 customer in server 1 and 2, re- 
spectively. In particular, pi = Pj +Pi 0,1 '. Since in steady 
state the amount of inflow is equal to the amount of outflow, 
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The solution of the last two equations is p n = 



n-3 

s H1+M2+M3 I ^ 3 
for n > 2. The values of {po,pi,P2} as a function of P3 
can be evaluated explicitly with MAPLE, by solving the 



first 2 s 



7 linear equations for {po, Pi 1,0 ' ', Pi"' 1 ' ', 



p(o.o,i)^ ^(1,1,0)^ ^(1,0,1)^ ^(0,1,1) y The resu ^j n g expression 
for F((j,i, fj,2, P3), however, is extremely cumbersome and not 
informative. 



Proof of Theorem [4] 

Since customers are randomly assigned to the available 
servers, F (fj,i, . . . , Atfc) is interchangeable. To see that F is 
differentiable in (/xi, . . . , A«k)> we note that F = ~Y^- np n 
where p„ is the steady-state probability that there are n cus- 
tomers in the system. In addition, {p n }J^ =1 are the solutions 
of a linear system with coefficients that depend smoothly on 
(m,...,(ih), andp n = (— T ±- i —)"- k p k for n > k - 1. This 
was shown explicitly for the cases k — 2 and k — 3; the proof 
for k > 3 is similar. 



Averaging principle for functions (proof of eq. 1111 ) 



Let (Fj(x)))j =1 be functions in the same function space T, 
and let e £ R. Let .R : (Fi, . . . , F k ) >-* R[F\, ...,F h ] £ R be a 
functional. We say that the functional R is interchangeable, 
if R(. . . , Fi , . . . , Fj , . . . ) = R(. . . , F j , . . . , Fi , . . . ) f or all * # j . 
We say that the functional R is differentiable if the scalar 
function R(e) := R[Fx = F + eHi, . . . , F k = F + eH k ] is twice 
differentiable at and near e = 0, for every F £ T and every 

(i^(x))* =1 e.F fc 

Given functions (Fj(x)))* =1 in T, denote F = ± Y^=i f j 
and Hj = Fj — F. By Taylor expansion, 



7?(6) = i?(0) + 6^|iiiJ J+ O(e 2 ), 



i=i 



where 4^ is the variational derivative. Because R is inter- 
changeable, 



A R 

fi( e ) = fl(0) + e |^-^^ + O(e 2 ) 



7=1 



In particular, if F = F, then X^=i = 0- Hence, -R(e) 



i?(0) + 0(e ), which is ITT 



Proof of Theorem [5] 

We first prove that F is differentiable. Denote 5^/ = 1 if indi- 
viduals i and i' influence each other, and 5iy = otherwise. 
For every k, every set of k consumers {i\,i2, ■ ■ ■ , ik}, and ev- 
ery increasing sequence of times < t% < • • • < t k , denote 
by P(ii,ti, %2, t2, . . . ,ik,tk) the probability that consumer ii 
adopts the product before time ti, consumer 12 adopts the 
product between times t\ and £2, etc., and all consumers who 
are not in {ii, . . . do not adopt the process by time t k . 
Then, 

P{i\,t\) = (1 - exp(— Pijti)) Y\ exp(-p^i). 

7#»l 



Similarly, 

P(ii,ti,h,t2, ■ ■ ■ ,ik,tk) — 

P(il,tl, il,t<2, ■ ■ ■ , ifc-l, tk-l) 

x ( 1 - exp I -(p ik + Si h>im qi m J(t k 



- tk-i 



X Yl eX P I _ I Pi + S i' i m- < Hrn J (** " *fc-l) 

iff{»i,...,i h } \ V m=l / 

Hence, the function P(ii, ti, 12, £2, • • • , ik, tk) is differentiable 
in {pi, qi}. Finally, 

1 M h 

W;te},fe})] = 5 EEra 

7T fc = l V 

x / / "/ P(ii,ti, ■ ■ ■ , ifc-i, t k -!,ik,t) dtk-x ■ ■ ■ dtx, 

where 7r ranges over all permutations on the set of M indi- 
viduals. Therefore, the differentiability of E[N(t; {pj}, {<}j})} 
follows. 

Because the network is translation invariant, F is weakly- 
interchangeable in {pj} and in {qj}. By this we mean that 

• If p m = p, pj = p for all j 7^ m, and qj = g for all j, then 
F 1 is independent of the value of m. 

• If q„ = q, qj = g for all j 7^ n, and Pj = p for all j, then F 
is independent of the value of n. 

Therefore, the result follows from a slight modification of the 
proof of Theorem [JJ 



Proof of equation 1131 



Since F is interchangeable, the quadratic term in the Tay- 
lor expansion of F(pi, . . . ./xj.) around the arithmetic mean is 
equal to 

k 



~ M.4)(Mj - Pa) 



d 2 F 



»,7=1 

d 2 F 



dp,id^j 



(1 



dfj,idfi2 



d 2 F 



dfiidfii 



A 4 i=l 



E (fJ,i - fJ,A)[fJ,j - (1a) + 

Since pyi is the arithmetic mean, 
^ (Mi - Ma)(a*j - Ma) = X^M* - Aa) 5Z(Mj - P-a) = 0. 

i,7=l »=1 7=1 

Therefore, the result follows. 
Proof of Lemma [3] 

Consider the case where fM = p, + h for i — 1, . . . , k. By 
equation |)13(| . 



1 

9 - P)(a*j - m) 



a 2 F 



i,J=l 

1 9 2 F 



A 



M 



2 dfiidfj,2 
On the other hand, since 

F(/Lt + ft, - . - , p + h) = Fhomog. (fi + h), 

we have 



kh z 



1 

9 z2 (Mi - A)(Mj - A«) 



<9 2 F 



i,7 = l 



Fhomog. (m) • 



A 
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Therefore, 



{po,Po,i,Pi,o, • • ■ ,Pi,6,Po,7} are 



2 d/j,idfi2 



Hence, 



h 

kh = — Fh' omog .(p)- 



d 2 F 



dfiidpi 



A 



homog 



d 2 F 

dfiidfii 



Calculation of (~X 

We illustrate the computation of the coefficient a for a queue 
with 8 servers. Consider then the case of a single server with 
service time fii, and seven servers with service time fi. De- 
note by po,n and pi,n, n = 1,...,6, the steady-state prob- 
abilities that n out of the homogeneous servers are busy 
and that the single heterogeneous servers is free or busy, re- 
spectively. The equations for the 2-8 — 1 = 15 variables 



Ap ,o = tfPo.i + AHpl.o , 

— f>o,o— +pi,o(A + fii) — pi,i[i = 0, 
8 

,, . 8 — n . , ,. 

PO.n (A + nfi) = po, n -i- A +po,n+i(n + 1)11, 



,6, 



Pi,n (^i + A + n/Lt) = pi, n -iA + pi, n +i(n + l)/i + 



A 



8-n 



P0,7(A + 7)/ti) = po,6- +P7flip, 

where p = j-^, Pn = p""V for n > 8, and Er=oP« = L 
These equations can be solved with Maple 0, and the solu- 
tion can be used to calculate F(pi, fx, . . . explicitly. Dif- 

ferentiating this expression twice with respect to /Lti, differ- 
entiating -Fhomog., see eq. p>|. twice with respect to /x, and 
using Lemma [3] yields equation [I14IJ . Substituting p, — 5 
and A = 28 gives a « 0.00837. In addition, X^i=i(/ li — p) 2 — 
e 2 ELi h2 = 71e2 - Therefore, aEtife - P-f « 0.594e 2 . 



2 The Maple code is available at www.bgn. ac . il/~ ariehg/averagingprinciple .html 
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